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Gradient and Passive Circuit Structure in a 
Class of Non-linear Dynamics on a Graph 

Herbert Mangesius, Jean-Charles Delvenne, Sanjoy K. Mitter 


Abstract 

We consider a class of non-linear dynamics on a graph that contains and generalizes various models from network 
systems and control and study convergence to uniform agreement states using gradient methods. In particular, under 
the assumption of detailed balance, we provide a method to formulate the governing ODE system in gradient descent 
form of sum-separable energy functions, which thus represent a class of Lyapunov functions; this class coincides 
with Csiszar’s information divergences. Our approach bases on a transformation of the original problem to a mass¬ 
preserving transport problem and it reflects a little-noticed general structure result for passive network synthesis 
obtained by B.D.O. Anderson and P.J. Moylan in 1975. The proposed gradient formulation extends known gradient 
results in dynamical systems obtained recently by M. Erbar and J. Maas in the context of porous medium equations. 
Eurthermore, we exhibit a novel relationship between inhomogeneous Markov chains and passive non-linear circuits 
through gradient systems, and show that passivity of resistor elements is equivalent to strict convexity of sum-separable 
stored energy. Eventually, we discuss our results at the intersection of Markov chains and network systems under 
sinusoidal coupling. 


I. Motivation 

Gradient methods provide an elegant way to physics motivated modeling 12 El and are closely linked to passivity 
theory and the circuit concept El a. They are a basic tool in studying and designing non-linear systems on a 
graph, e.g., in distributed optimization El or in multi-robot problems such as coverage or formation control, cf., 
e.g., El, El, and references therein. 

Another pillar in network system studies is the classical consensus problem 0. An equivalence between the 
dynamics (trajectories) of Markov chains and consensus networks has been source of recent advances in consensus 
theory a. For LTI symmetric consensus networks such an equivalence has been linked to the averaging dynamics 
of unit capacitor RC circuits in El chap. 3. Within the mathematics community, a static relationship is usually 
considered between Markov chains and electric circuits (resistor networks) Eol. The static (algebraic) circuit 
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equations due to Kirchhoff and Ohm in fact are known to serve as generic structure underlying various scientific 
and computational problems, see, e.g., ini chap. 2. 

Gradient formulations of Markov chains using sum-separable energy functions have been of recent interest in 
dynamical and non-linear systems Cl CH Cl ISl. Interestingly, sum-separability of energy is an axiom in 
interconnected dissipative systems H and has origins in the circuit concept. 

In this paper we bring these various concepts together in novel ways, based on a gradient structure for a class 
of non-linear dynamics on a graph that covers a wide range of prominent network system problems. 


II. Problem description and related literature 

Let G = (TV, B, w) be a weighted directed graph, where N = {1, 2,..., n} is the set of nodes, i? = {1, 2,..., 6} C 
N X N denotes the set of branches whose elements are ordered pairs (j, i) denoting an edge from node j to i, 
and w : B ^ ]R>o is a weighting function, such that w{{j,i)) =: Wij, if (j, *) € B, else Wij = 0. Associated to a 
graph is the Laplace matrix L, dehned component-wise as [L]y = —Wij, [L,]ii = Wij. For strongly connected 
graphs, denote the positive left-eigenvector associated to the unique zero eigenvalue of the Laplacian by c, and 
dehne C := diagjci, C 2 ,..., c„}. 

An important generalization of the symmetry condition on Laplacians that L = is the particular type-symmetry 
that for some C, and i,j G N, 

CiWij = CjWji CL = L^C. (1) 

Equation ([U is known in the literature on Markov chains as detailed balance, or as reversibility w.r.t. c, cf., M 
chap. 2. 


We consider the general class of dynamics on a graph G described component-wise by an ODE of the type 


ti= 


ij (j){xj,Xi), iGN, 


( 2 ) 


where is Lipschitz continuous, 4>{a,b) negative if a < 5, zero iff a = b, positive if a > 6, and \(j){a,b)\ is 

increasing if |a — 6| is increasing. 

The class (H includes many known network models: The usual linear consensus system fS) is obtained from 
setting (j){xj,Xi) = Xj—Xi. If <j){xj,Xi) = f{xj — Xi){xj — Xi), with f{z) = f{—z) > 0, then, the ODE Q describes 
a continuous-time opinion dynamics anl. Eor instance, one may choose f(z) = | tanh(p ■ z)\, p > 0, which is 
a good choice for modeling saturation phenomena in the interaction. If (j){xj,Xi) = ip{xj — Xi), ip{z) = —'tp{z), 
then we recover the non-linear consensus class introduced by Olfati-Saber and Murray in im, with Ip = sin a 
prominent instance. Beyond the presented known interaction ^pes, our model also includes couplings of the form 
(p{xj,Xi) = g{xj) — g{xi), where g is an increasing functioiu e.g., ln(a:), e^, p > 0, on the respective domain 


*or = l{xi) — l{xj), where Z is a decreasing function 
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of definition. The latter interaction type covers a discrete version of an equation system that models the non-linear 
diffusion of a gas in porous media, see 112 (and mi for the continuous context). 

From an operational point of view, we are interested in bringing the ODE system (|2, under the assumption of 
detailed balance, into the gradient form 

q = - Kiq ) VE { q ), (3) 

where q is a suitable transform of the original state x, K(-) is a symmetric, positive semi-dehnite matrix function 
that inherits the sparsity structure of the graph G, and E { q ) is a sum-separable Lyapunov function. 

This structure dehnes gradient descent systems living on subspaces of R", where — Vi?(-) •K(-)Vi?(-), describing 
locally the dissipation rate of E , is negative definite. In the context of gradient systems this structure is quite 
particular, as we impose the sparsity constraint given by G, require sum-separability of the potential E , and do 
not require positive dehniteness of the inverse metric K; these constraints are not usual from a classical gradient 
system point of view, cf., e.g., Il20l . but turn out to be elementary in a passive circuits context. 

For particular cases of graph weightings and functions (p, gradient structures for the class (|2 have been established; 
For symmetric consensus systems, (i.e. L = L^, ,Xi)= xj —Xi), it is well known that the network dynamics are 

a gradient descent of the (non-sum-separable) interaction potential ^x^hx, see, e.g., il.Inllll a port-Hamiltonian 
view as gradient descent of the sum-of-squares energy ^ presented. Under the less restrictive assumption 

of detailed balance weightings, the linear system dynamics (understood as Markov chain) has been formulated as 
gradient descent of free energy, resp. of relative entropy, in the works ifTsll and IITtII . In lfT2 . for systems with 
detailed balance weighting, and (j>{xj,Xi) = g{xj) — g(xi), g increasing, a smooth gradient descent structure is 
presented for sum-separable energies CiTf(a;i), H being strictly convex and smooth on R>o- For the particular 

non-separable interaction case of having sinusoidal coupling, but symmetric weighting, gradient flow structures are 
represented, e.g., in ll^ or ll2^ . where energy functions however are non-separable. 

In the following we solve the general gradient representation problem and motivate the proposed structure 
requiring sum-separable energy functions from a passivity and circuit systems viewpoint. 

III. Gradient representation 

With the following result we provide a procedure to bring a dynamics (|2) into the form Q. By that we characterize 
a family of sum-separable Lyapunov functions characterizing asymptotic stability of agreement states, i.e., states 
where all components are equal. 

Theorem 1. Consider a network system dynamics governed by the protocol © on a strongly connected graph G 
such that detailed balance O holds for some C. Define the new state q := Cx, and consider the sum-separable 
function 

E{q) 

i£N 


(4) 
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where H : R —>■ K. w any -function, and set h{z) := If H is strictly convex, then the system can be 

represented as 

q = -K{q)VEiq), (5) 


where K{-) is defined as the irreducible and symmetric Laplace matrix having components 

h{xj)-h{xi) d J T I-’ 


[Kh ■■= 


( 6 ) 


The function © is a Lyapunov function establishing asymptotic stability of the equilibrium point Xool, with 
equilibrium value the weighted arithmetic mean x^o = 

2^i€N 


Proof: First, we observe that by the chain rule with Cj ^qi = Xi, 

-^E{q) = = c^h{xi)c~^ = h{xi). 


The network dynamics © can be written equivalently 


—c^Xi= ^ Wij(j){xj,Xi) qi = 


CiWijlj){Xj,Xi) 






Expanding by h{xj) — h{xi) yields 


r.it^^B h{x,)-h{x.) 




j:(j,i)eB 

^ q = -K{q)VE{q), 


(7) 

( 8 ) 

(9) 

( 10 ) 

( 11 ) 


where we use the identity J2j-.{j,i)GBi^]ij = “Mm- 

Next, we show that the matrix K is a symmetric, irreducible Laplace matrix. As H is strictly convex and of type 
h, as derivative of H, is an increasing function, i.e., for any two real numbers a, b, h{a) < h{b), whenever a < b. 
We observe that (j){a, b) = sgn(a — b)d{a, b), where d is a Lipschitz continuous distance in R. Now, is 

symmetric in both arguments and positive for non-identical arguments, as 

sgn(xj - Xj) • d{xj,Xi) _ sgn(xj - Xi) ■ d{xj,Xi} 


h{xj) - h(xi) sgn{xj - Xi)\h{xj) - h{xi)\ 

dixi,Xj) 


( 12 ) 

(13) 


\h{x^) - h{xj)\ 

where we used the fact that a distance is symmetric in both arguments and positive. By hypothesis ©, for all i j, 

CiWij = CjWji, so that 


— CiWi'j 


ip{Xj,Xi) 


(l>iXi,Xj) 


= -[K],i, 


(14) 


h{xj) — h{xi) ^ ^ h{xi) — h{xj) 

and hence, K is symmetric. It is also irreducible, as the underlying graph G is assumed to be strongly connected. 
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Further, for Xj —)► Xi the components — [K]jj are well-defined in the sense that their value remains positive and 
finite: For two real numbers b,a, b > a, 


lim 


(/)(&, a) 


Slim 


d{b, a) 


b->-a h[b) — h{a) b^a \h{b) — h{a)\ 


=lim .. > 0, 


(15) 

(16) 


fc-j-a \b — a| \h{b) — h{a)\ 

as the distance is Lipschitz continuous and increasing the further one moves away from a, and h is an increasing 
function, so that the second fraction is positive as well. Hence, [K]ij-elements are finite and positively bounded 
away from zero for all parametrizations. 

Convergence to an agreement state Xool that is asymptotically stable follows from LaSalle’s invariance principle: 
Stationarity implies 0 = K{q)VE{q), which is the case if and only if VE{q) G ker(K) = span{l}, as K is 
a symmetric, irreducible Laplace matrix. Then, as K is positive semi-definite, E will decrease its value along 
solutions, as 

(17) 


E{q) = -VE{q) ■ ^{q)VE{q) = -11Vi?(q) 11|. 


until a minimum is reached when VE{q) = h{x) G spanjl}. The function h is an increasing function, hence 
bijective, so that the stationarity condition at a point x is equivalent to i G span{l}. Given an initial condition 
from a set defined by qi = const., the set I := {q G M", Qi = const. : E{q) = 0} is a singleton, as = 
l^K{q)yE{q) = 0, and span{l} HI is a point Xool- This shows that £’ is a Lyapunov function for the considered 
network dynamics establishing asymptotic stability of an agreement state Xool- The agreement value results from 
Si di{t) = const., so that CiXi(0) = x^c Si therefore Xoo = the weighted arithmetic mean. 


In the following we provide a passive circuit interpretation of this gradient result, from where an intuitive meaning 
of the q-variable system follows. We then present for our gradient construction the relationship to a structure result 
in passive network synthesis and show an equivalence between strict convexity of E and passivity of resistor 
elements. 


IV. Synthetic circuit structure and passivity 

A. Circuit formulation 

In circuit theory the dynamical behavior of a system is seen as the result of the interaction of a finite number 
of interconnected circuit elements. Circuit elements are single-input-single-output systems among which lossless, 
dynamical ones that can store energy and possess memory, e.g. capacitors in electric circuits, and memoryless, 
non-dynamic ones that dissipate energy, e.g. resistors, play important roles. Sum-separability of stored energy in 
this context thus arises naturally, as it is the sum of energies stored in individual lossless circuit elements. 

We consider the lossless, dynamical multi-input-multi-output system 


q{t) = UN{t) 

VNit) = h{x(t)) = VE{q), 


( 18 ) 
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which is an input-output model for elastic systems Q. In the circuits context it describes a generalized capacitor 
bank, where the vector q collects charges in n capacitors, each having a capacitance ci, so that ttjv is a vector of 
input currents, r/jy ^ vector of generalized output voltages, and E is the energy stored in the generalized capacitors. 

Example 1 (Passive LTI capacitor and electric energy). Electric energy stored in n LTI passive capacitors is given 
by 

.. n 1 

E{q) = 2 X! = 2 ^ (19) 

^ i=l ^ i=l 

which is of the particular form proposed in Theorem [T] The gradient of electric energy w.r.t. charge as state is 
VE{q) = vc, the capacitor voltage vector serving as output. Capacitors are current controlled, i.e., um = ic, ic,i 
the capacitor Ts input current, so that we obtain the well-known input-output characteristic sCvc = ic^ with s a 
Laplace variable, and the state space representation q = ic, vq = VE{q). 

In a resistor network where b resistor elements (representing the branches in a graph) are interconnected at n 
nodes, the governing equations follow Kirchhoff’s voltage / current law (KVL / KCL) and Ohm’s law HD chap. 
2; Let B be the usual n x b branch to node incidence matrix, and the b x b diagonal matrix collecting the 
positive values of b conductances (inverse of resistances). The vector vb collects the b resistor voltages, is the 
corresponding b resistor currents, ijv is the vector of node currents and the vector of node voltages. We then 
have 

KVL: Ohm: is'= 03 ^ 3 , KCL: ijv = Big. (20) 

Setting Db = diagjrj"^,..., r^^}, with positive resistances at an edge e = {j, i) given by 

„ . 1 h{xi)-h{xj) 

aWij (l){Xi,Xj) 

the gradient system proposed in Theorem ([T]) represents an i?C-circuit with dynamics described by voltage variables 
at the network nodes as 

= q =-KVL;(q) =-BDbB^vat, (22) 

where we use the fact that a symmetric, irreducible Laplacian can be factorized as K = BDbB^. This synthetic 
structure of a non-linear network dynamics (|2|i as i?C'-circuit is depicted in Fig. [T] 


Example 2 (LTI RC circuit and consensus dynamics). Consider a resistor network where at each node with one 
terminal an LTI capacitor of capacitance Ck is connected, see Ex. [T] Kirchhoff’s and Ohm’s laws lead to a local 
balance equation for currents flowing in and out at each node k £ N: 


ic,k 


O Vk 


e=U,k)eB ees ® 3.U,k)GB ® 


E 

j:(j,k)GB 


1 

Cfc T }ij 




VC,k), 


VC,k) 


(23) 

(24) 
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Fig. 1: Output feedback representation of the gradient dynamics as circuit: Resistor network in Kirchhoff-Ohm 
factorized form (red), capacitor bank (blue) 


where we used the capacitor equation vc,k = c^^ic,k- Define the Laplacian L, with , and = 

J2j^k cjrk • Then, the averaging iiC-electric circuit dynamics of terminal voltages are a detailed-balance LTI 
consensus dynamics vq = —hvc- 

Remark 1. We note that the weights Wkj = —-— in Ex.|2]have a (physical) meaning in RC circuits: They represent 
inverse time-constants, or cut-off frequencies. 


B. Hill-Moylan-Anderson result, passivity and convexity 

D.C. Youla and R Tissi showed in Il24l that a synthesized dynamics that solves the classical LTI network synthesis 


problenj^ is structured as (negative) feedback system, in which a dissipative and memo^less network controls a 
lossless and passive system that is comprised of decoupled unit capacitors and inductor^. 

Anderson and P.J. Moylan in 1251 and Hill with Moylan in ll27l proposed a structure result for non-linear ODE 
systems, which characterizes a realization of a vector field as lossless-memoryless decomposition similar to the 
well-known Youla-Tissi one for linear networks. Eor an equation system x = f{x) that generates solutions along 
which E : R" —K is a differentiable Lyapunov function, their non-linear analogue to extract a lossless and 
dynamic part leads to an output feedback structure as illustrated in Eig. |2] where : K” —R" denotes any 

(generally non-linear) function such that [(Vi?)“^ o VL](£c) = x. 

The preceding circuit formulation of the gradient result in Theorem [T] yields an explicit implementation of this 
general structure result for the non-linear ODE system class (|2]l, i.e., for [f{x)]i = * S A^, 

where the inversion mechanism is realized by interconnected passive resistor elements: The Laplacian structure 
resulting from Kirchhoff’s laws allows to simply multiply pairwise interactions in the non-linear vector field with 
the reciprocal of energy gradient differences, to obtain the conductances 




4>{Xj,Xi) 




{hi) e B, 


(25) 


^Classical network synthesis is concerned with reproducing a prescribed LTI input-output behavior in terms of a finite number of elementary 
passive, linear, ideal (lumped) circuit elements and a scheme for interconnecting them du m 

^ The procedure to obtain this feedback representation is commonly referred to as reactance extraction, as the reactive, i.e., dynamical network 
elements are extracted from the composite system. 
















































Fig. 2: Hill-Moylan-Anderson decomposition for an ODE system x = f{x). 


with Wij, Ci, (j), E as specified in section [H] and Theorem [T] In this reciprocal inversion mechanism, strict passivity 
of a resistor element is equivalent to positivity of resistance (resp. of conductance) and hence to dissipation of 
energy across a resistor edge connecting two non-identical potentials. In the context of the system class (O, we 
observe an interplay between strict passivity of resistor elements and strict convexity of sum-separable energy; 

Proposition 1. Each resistor element characterized by a conductance (I251 l is strictly passive if and only if the 
sum-separable energy E as given in Theorem Q] is strictly convex. 

Proof: Strict convexity of E is equivalent to increasingness of the gradient component functions h. If h 
is increasing, then each = CiWij > 0, see ( [13] ) with the preceding arguments. If 

h were not (monotonously) increasing, but also decreasing at some part of the state space, then, in that region 
(p(xj,x ) ^ non-linear resistance would not be passive, but active. Conversely, if h is constant on 

some interval on the real line, then resistance would vanish, and so does dissipation and the dynamics, hence the 
resistance is not strictly passive (neither active). ■ 

For sum-separable, strictly convex energy to be a Lyapunov function, passivity of each (internal) resistor element 
is not necessary but only sufficient. It is required that K(q) is positive semi-definite, so that 

^E;(g) = -VE{q) • K(g)VE;(g) = -\\VEiq)\\l < 0 (26) 

except at the equilibrium point. This dissipation inequality can be satisfied when E is not strictly convex, i.e., 
individual non-linear resistors may locally (in some region on state space) be active (have negative resistance), as 
long as globally more energy is dissipated than created. 

Remark 2. In Hill, Moylan, Anderson’s work it is assumed that the function exists. Moylan, in li28l chap. 

10, conjectures that convexity of i? is a sufficient condition for the existence of the function inverse. Strict convexity 
is in fact sufficient as a classical result from convex analysis shows: The function inverse = VE*, where 

E* is dual to E in the sense of Young, see, e.g., Ii29l . chap. 3. 

C. Conservation of charge and Markov dynamics 

A pillar of the circuit concept is the conservation of total charge, see, e.g., Ii30l chap. 1. In the following we 
relate the non-linear charge dynamics associated to the class (|2]) to the dynamics of a (spatially) inhomogeneous 


Markov chain. 
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With K(-) being a symmetric, iiTeducible Laplace matrix for all parametrizations, ker(K(-)) = span{l}, so that 
= const, for all times f > 0, since = 0. This recovers the conservation principle for charge in our 
circuit interpretation of Theorem[T] Without loss of generality, we can choose q(0) G such that (7i(0) = 1. 
The normalized q-vector then also has the interpretation of a probability mass distribution on a discrete probability 
space: Each node i G TV is a possible state, and qi (t) is the probability of a random walker on a graph G of being 
in state i at time t, see, e.g., El- 

The equation system describing the probability transport associated to the non-linear averaging dynamics (|2]) in 
a;-variables follows from the gradient formulation in Theorem [T] with the admissible choice H{x) = ^x'^, so that 
VE{q) = x: 

Define F^(-) := K(-)C”^ and observe that F = C~^K is an irreducible (non-symmetric) Laplace matrix satisfying 
detailed balance. Then, 

q = -K{q)x = -K{q)C~^q = -¥^{q)q. (27) 

where F^ is the infinitesimal generator of a (spatially inhomogeneous) Markov chain. This Markov chain asymptotically 
reaches the invariant probability measure given by the normalized capacitances c with ~ 

This equivalence between passive /JC-circuits and Markov chains as dynamical systems bears the following 
novelties: 

i) The usual relation between electric circuits and Markov chains in the applied mathematics literature restricts 
to a static equivalence between a resistor network and the probability transition kernel of the Markov chain, cf., 
the seminal work m and references therein. In the engineering literature a dynamical relationship is known only 
for LTI symmetric consensus systems, which are equivalent to homogeneous, symmetric Markov chains and unit- 
capacitance RC-circuits, see, e.g., [O chap. 3. We extend those results to dynamical, non-linear TJC-circuits, where 
we show the relationship between detailed balance and non-unit capacitances. 

ii) With capacitances c normalized such that Ci = 1, stored energy E{q) = CiH{c~^qi), El strictly convex, 
corresponds to the class of information-divergences of a probability distribution q to the equilibrium distribution 
c, introduced by Ali and Silvey and Csiszar, cf., |[33]. The usual technique to prove decreasingness of E is 
based on Jensen’s inequality 1341 . see also llT5l . Theorem [T] establishes this dissipation inequality for the class of 
Csiszar’s information divergences in a novel way, namely by exhibiting a passive RC circuit structure. 

Remark 3. Other physical interpretations of the q-variable (charge, probability) and a;-variable (voltage, density) 
system representations can be found in mass action chemical reaction networks, see, e.g., ll^ and OS. 

V. Discussion 

A. Coupled oscillator models and electric power grids 

A generic model in the study of phase-coupled oscillator networks is given by the ODE system on a graph G, 

= uji+ ^ Wij sm{9j — 9i), i G N, 


( 28 ) 
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where u) = (wi, W 2 ,... ,ujn) S K” is the vector of natural (driving) frequencies, and the state 0 S T" is an n-vector 
of angles as elements of the n-Torus. 

If we set Wij = ^, K > 0 for all {j,i) € B, then (l28l l represents Kuramoto’s oscillator model llJTl . If 
Wij = a voltage magnitude, yij the complex admittance of a line (j,i), and > 0 a damping 

parameter, then (l28T l describes a so-called droop control setup for frequency stabilization of generators in an electric 
power grid whose diffusively coupled voltage angles 9i are driven by nominal power inputs uji, see, e.g., f22\ . 

Observe that the detailed balance condition ([T]i naturally applies in this setting: In Ex.|2]we saw that CiWij = CjWji 
implies that weights Wij have the form Wij = 7 ^, = Vji. Following this RC circuits view, we may take 

= ^{uij) having unit siemens (one over ohm), and c~^ = |ui| |/Z3i. Capacitance has unit farad F = 
so that Di should carry the unit volt-ampere-seconds (an energy), which is similar to a measure of a power deviation 
per base frequency. This indeed matches the meaning of the factor Di in the droop control setting. 

Using our reactance extraction approach we can write (l28l l as (driven) gradient system, with q = CO, ujc ■= Cu), 
and the gradient h{6) — VE{q), 

q = u;c-K(0)Vi?(q), [K],, = (29) 

For instance, if we choose E = ’ (electric energy), then, = sinc(0j — di), the sine 

cardinalis, or sampling function. We denote the corresponding inverse metric by Ksinc. Let u)c = cl, c G M, so 
that without loss of generality we can study the dynamics in a rotating frame at speed c and set a; = 0 in (|29] | 
||2^ ll^ . Phase synchronization takes place if maxi^g^v \0j — 0i\ < tt, because in that case sinc(-) > 0, i.e., 
all non-linear resistances are passive, so that limt^oo 0{t) Bool, according to Theorem [T] By that we 

recover a known phase synchronization result, see, e.g., lf 22 l . but via a passive circuits approach. 

As shown in section imi we can bring the coupled oscillator model (l28l) (in uniform rotating frame) into the form 
of a Markov chain dynamics. The fact that the class of information divergences are Lyapunov functions, allows to 
explore mixing time bounds in Markov chains for convergence bounds in dynamics on a graph, which often are 
tighter than usual bounds based on the second largest eigenvalue of a Laplace matrix K, see lf39l for an overview. 

B. Discrete De Bruijn 's identity 

Two elementary quantities in information theory are differential entropy, a measure of the descriptive complexity 
of a random variable, and Fisher information, a measure of the minimum error in estimating a parameter from a 
distribution. Let S' C R be the simport set of a random variable X of finite variance, and let x{^) > 0, G S, be 
a probability density distributiorj^ for X. The differential entropy then is defined as 1401 chap. 9, 

Ent(-A) := — f a;(^) lna:(^)d^. (30) 

Js 

the gradient of the cumulative probability distribution function on S 
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The Fisher information of the distribution of X can be defined as in ||4^ chap. 16.6, 

J(X) := d^ = (31) 

This is a special form of the Fisher information, taken with respect to a translation parameter on the continuous 
support S, which does not involve an explicit parameter as in its most general definition PTOl chap. 12, see also 

ED. 

De Bruijn’s identity provides a relationship between these two quantities; For a process Y = X -\- \/tZ, where 
Z is a normally distributed random variable, 

^Ent(r) = \j{Y), (32) 


see Il40l Theorem 16.6.2. 

In the context of porous medium equations, Erbar and Maas in ifT^ propose the discrete version of Fisher 
information 


J{x) 


1 ^ CiWij(j){xj,Xi) {h{xj) - h{xi)), 

i,j&N 


(33) 


where (j){xj,Xi) = g{xj) — g(xi), g an increasing function. Note that the gradient of a function on a discrete space 
{N,B,w) is given by the (weighted) difference of the function values at connected nodes. 

For instance, if we choose g{xj) — g{xi) = Xj — Xi, and relative entropy E{q) = Ciqi In ^ as energy, so that 
h = In, we get 



-Ji^) = 9 X! CiWij\gm{xj,Xt) |lna;j - Inxi] 


i,jeN 


(34) 


where \gm{xj,Xi) := logarithmic mean of two positive variables, and | \nXj —\nxi\^ is the discrete 

equivalent to jVlnxp in OTT l. 

In the definition of Fisher information this gradient is integrated w.r.t. a;d^. On a discrete space gradients live on 
the set of edges B, while the density vector x is defined for elements indexed in the set N. The logarithmic mean 
accounts for this lack of support in the discrete case; By the mean value theorem, there exists a value Xij G [xi,Xj\, 
(where we suppose that density components satisfy Xi < Xj), such that 


In a;, —In 

Vlnxij = - - - O Xij ^ \gm[Xj,Xi). (35) 

^ J 

Further, lgm~^(a:i, Xj) = ^ ll42l . so that an “edge density” Xij can be seen as a (convex) interpolation 

of the density across edges e = {j,i) G B based on knowledge of density components Xi,Xj defined on nodes 
i,j G N. 

Let us consider the discrete De Bruijn inequality for a system |2] with coupling (j){xj,Xi = sin(xj — Xi). Then, 


sin(a;j — Xi) 
2 ^ ‘ Ina;,-— Ina;^ 

i,j^N 


Jix) = 9 XI 


|lna;j — Inc 


(36) 
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Using a discrete chain rule we can write the corresponding edge density Xij = “ sinc(xj — 

Xi)\gm(xjTXi). This is the logarithmic mean modulated by a sampling function kernel that takes values between 
zero and one (on sets where each edge corresponds to a strictly passive resistor). 

As far as we known, this connection between De Bruijn’s identity in information theory and the dissipation 
equality (l3^ as discrete version of it is novel. A discrete version on domains characterized by graphs is natural 
in applications, where high-dimensional data actually resides on nodes of graphs, and (non-linear) weightings may 
characterize an application’s peculiarity in terms of the irregularity of the domain. It would be interesting to further 
understand the role of discrete instead of discretized continuous information inequalities, as the presented one of 
De Bruijn, within the emerging field of signal processing on graphs B3l . 

VI. Conclusion 

In this paper we established for a general class of non-linear dynamics on a graph with detailed balance weighting 
a family of gradient structures associated to sum-separable strictly convex energy functions. This structure extends 
known gradient results in dynamical systems to pairwise couplings involving non-separable non-linearity. Based on 
our gradient formalism we made several connections between previously separated results in dynamical network 
systems, Markov chains, information and circuit theory, where at the heart lies a little-noticed structure result for 
non-linear network synthesis, due to B.D.O. Anderson, RJ. Moylan and D. Hill. 
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